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[57] ABSTRACT 

An internet activity analyzer includes a network interface 
controller, a packet capturing module, a packet analysis 
module, and a data management module. The network 
interface controller is connected to a transmission medium 
for a network segment and is arranged to receive the scream 
of data packets passing along the mediuno. The packet 
stream is filtered to remove undesired packet data and is 
stored in a raw packet data buffer. The packet data is 
decoded at the internet protocol layer to provide information 
such as timing and sequencing data regarding the exchange 
of packets between nodes and the packet data for exchanges 
between multiple nodes may be recompiled into concat- 
enated raw transaction data which may be coherently stored 
in a raw transaction data buffer. An application level proto- 
col translator translates the raw transaction data and stores 
the data in a translated transaction data buffer. The translated 
data provides high level information regarding the transac- 
tions between nodes which is used to monitor or compile 
statistics regarding network or internetwork activity. The 
data management module communicates with the packet 
capturing module and the packet analyzer and, particularly, 
the data in the raw packet, decoded packet, raw transaction, 
and translated transaction data buffers to provide real time 
and stored analytical information concerning internet activ- 
ity. 

23 Claims, 12 Drawing Sheets 
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APPARATUS AND METHOD OF ANALYZING 
INTERNET ACTIVFTY 

BACKGROUND OF THE INVENTION 

1, Field of the Invention 

The present invention relates generally to computer net- 
works. More particularly, the present invention relates to an 
apparatus and method for analyzing internet activity. Still 
more particularly, the present invention relates to an appa- 
ratus and method of monitoring packets to rate Internet 
usage and monitor Internet performance. 

2. Description of the Related Art 

An internet may be defined as a collection of networks 
interconnected, for example, by routers which allow them to 
function as a single virtual network. For example, the 
Internet (capital "F) is the largest internet in the world and 
comprises large backbone nets (such as MILNET and 
NSR^fET) and various regional networks. The popularity of 
the Internet has made it increasingly important to be able to 
monitor the use of services offered and rate those services 
accordingly. For example, objective information such as 
which Internet services are being used, who is using them, 
how often they have been accessed, how long accesses have 
been and what browsers have been used is needed. 
Additionally, remote access by selected users to generate 
reports in real time and for access to historical Internet 
activity data based upon such information is needed. Finally, 
an Internet activity monitor which can provide alarms to 
notify selected users of network or site problems is needed. 

Conventionally, selected network activities may be retro- 
spectively analyzed by reviewing server log files. Referring 
now to FIG. 1, a conventional server 10 is shown in 
communication with a network transmission media 40 such 
as a twisted pair or coaxial line. A log file 30 is typically 
recorded in server memory 20 and comprises a set of data 
concerning server interactions with other network nodes. 
Log file analyzers must access this data and analyze its 
contents to determine statistics about the server. 

However, while some useful data may be gleaned from 
server log files, log file analyzers are inadequate for rating 
network usage and performance for a variety of reasons. 
First, thae is not a universal standard for log file formats. 
Thus, typical log file analyzers function only for particular 
servers. Additionally, since log files are independently kept 
by servers, there is no easy way to organize and track data 
from multiple servers on one segment or to track the usage 
of a network by users with no server on the immediate 
segment. The maintenance of a log file can also impact the 
performance of the server and, therefore, the network since 
server resources are consumed by log file processing. As 
more log file detail is demanded for network activity 
analysis, ao increased drag on server resources such as 
processing time and storage will be experienced. 
Additionally, log file analyzer results may be compromised 
since the data in the log file is under the control of the server 
administrator and nmy be edited or altered prior to analyzer 
access and treatment. Finally, since log files are typically 
accessed after usage, real time analysis is in4>ractical. 

Specialized software server hooks in the form of separate 
software programs executed by the server may also be used 
to monitor server activities, but, since these books are 
typically particular to their server and server access is 
required for their analysis, they suffer the same problems as 
log file analyzers. 

What is needed, therefore, is an internet ratings and 
performance analyzer which is not reliant upon server 
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resident information such as log files and which, 
accordingly, allows the user to objectively measure and 
analyze network activity, customize the type of data that 
collected and analyzed, undertake real time analysis and 
5 receive timely notification of network problems. 

SUMMARY OF THE INVENTION 

The present invention overcomes the limitations and 
shortconmgs of the prior art with an apparatus and method 

10 for analyzing internet activity. 

The internet activity analyzer includes a network interface 
controller, a packet capturing module, a packet analysis 
module, and a data management module. The network 
interface controller is connected to a transmission medium 

15 for a network segment and is arranged to receive the stream 
of data packets transmitted over the medium. The packet 
capturing module is in communication with the network 
interface controller and filters the packet stream to remove 
undesired packet data and passes the raw packet data to a 

20 raw packet data buffer. The raw packet data is decoded at the 
internet protocol layer to provide information regarding the 
packet This information includes administrative data such 
as timing and sequencing data regarding the exchange of 
packets between nodes. After such decoding, a plurality of 

25 packets from a plurality of exchanges between a plurality of 
nodes may be recon^>iled into proper order using the admin- 
istrative data to provide concatenated raw transaction data 
whidi may be coherentiy stored in a raw transaction data 
buffer. For example, a first exchange between first and 

30 second nodes and a second exchange between third and 
fourth nodes may be simultaneously processed by the ana- 
lyzer. The data in the raw transaction data buffer may be 
sorted and filtered by the user to organize and eliminate 
undesired data. An application level protocol translator 

35 translates the raw transaction data and stores the data in a 
translated transaction data buffer. The translated data pro- 
vides high level information regarding the transactions 
between nodes which is used to monitor or con^ile statistics 
regarding network or internetwork activity. The data man- 

40 agement module communicates witfi the packet capturing 
module and the packet analyzer and, particularly, the data in 
the raw packet, decoded packet raw transaction, and trans- 
lated transaction data buffers. The data management module 
includes a logical unit to provide calculated information and 

45 an inference analyzer to provide data inferred from the data 
in the data buffers. The data management module also 
includes modules for analyzing and stripping the data from 
the data tHiffers into smaller portions and indexing the 
stripped data to further conserve storage space. 

50 By capturing and analyzing data packets transmitted 
along a network transmission medium between nodes, the 
internet activity analyzer of the present invention facilitates 
the analysis of performance based upon reliable data without 
requiring access to the server. This access is transparent to 

55 the network and facilitates reliable data acquisition without 
draining server resources. Since the analyzer may be sepa- 
rate from the server, the data may be gathered without 
requiring server input and the opportunity to taint the results 
of an analysis. The filtering of data may be arranged such 

60 that data from selected nodes or protocols may be acquired 
Thus, data regarding a site with multiple servers may be 
accumulated and a comprehensive site analysis may be 
provided without requiring s^arate accesses to the various 
servers which comprise the site. Additionally, a group of 

65 users who access the network without passage through a 
single server may be analyzed, such as, for exa^^)le, those 
In a college computer lab. 
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Timing and sequencing information and ratings data may DESCRIPTION OF THE PREFERRED 

be provided through analysis of pacicet and transaction data. EMBODIMENTS 

Since the analyzer is airanged to acquire packets as they Packets are a basic unit of network comraunication. When 
traverse the network media, real time analysis of the infor- iro^i^cia a uaw^ umi ui utiwvi^j^iuiuumvawv^w. ..^» 
mation is facilitated. This allows users, whether local or 5 « "^^^^^^ ^<>^^ coitunumcates with another node on the 
remote, to view network performance and analyze the network or through the network, a number of packets are 
accesses to selected sites as they occur, and, similarly, exdianged between the two nodes. By monitoring the flow 
facilitates timely notification to users concerning network of packets over network media, information about the trans- 
problems. The provision of applications level decoders actions between the nodes may be obtained. In an Internet 
facilitates decoding the data packets into high level infor- application, where multiple networks are interconnected, the 
mation and, therefore, allows complex profiles to be devel- exchange of information between a first node and a second 
oped regarding the transactions iKtween selected nodes. node may be obtained firom packets traversing a network 

„^ ^ ™^ . »^ rrvT^o media on a given network segment provided that at least one 

BRIEF DESCRIPTION OF THE DRAWINGS j^^^i^^ ^ ^ ^^^^g, i5 ^„ 

FIG. 1 is a block diagram illustrating a conventional segment The packets traversing network media may be 

server log file. observed and aq>tured by the internet activity analyzer 2(W, 

FIG. 2 is a block diagram illustrating the flow of data in 300 of the present invention without interaction with the two 

a preferred embodiment of the internet activity analyzer in nodes in the transaction. 

accordance with the present invention. Referring now to FIG. 2, a block diagram of an overview 

FIG. 3 is a block diagram illustrating a preferred erabodi- of the data flow for a preferred embodiment of the internet 

ment of the internet activity analyzer constructed in accor- activity analyzer 200 in accordance with the present inven- 

dance with the present invention. tion coni^nses a network interface 240 which is coupled to 

FIG. 4t3 is a block diagram illustrating a preferred a network via a transmission medium 205 such as a twisted 

embodiment for a memory of the internet activity analyzer pair or coaxial line. Hie network interface 240 ou^uts to a 
in accordance with the present invention. ^ packet capture section 230 which is coupled to a packet 

FIG. 4b is a block diagram illustrating a preferred analysis section 220. The packet analysis section 220 out- 
embodiment for a packet capturing module for the internet puts to an alarm generation section 270 and is coupled to a 
activity analyzer in accordance with the present invention. data management and storage section 210. A report genera- 

HG. 4c is a block diagram illustrating a preferred cmbodi- section 250 is coupled to the data management and 
ment for a packet analyzer of the internet activity analyzer 30 storage sccUon 210 and a report and sUtistics retrieval 

in accordance with the present invention. section 260, A security interface 280 is coupled to the 

FIG. 4d is a block diagram illustrating a preferred network interface 240. the alarm generation section 270 an^ 

embodiment for a data management module for die internet the report and staUsUcs remeval section 260 The sectmty 

activity analyzer in accordance with the present invention. ^^^^^^ ^ accessed, for example, by a phone hne 295. 

FIG. 5a is a lock diagram illustrating a conventional raw 35 Generally, packets comprise a pluraUty of fields, estab- 

packet data profile lished by syntax, such as a header portion and a data portion. 

FIG. Sb a block diagram illustrating an exemplary The informaUon in the packet ty^ 

embodiment of a daU table derived from data in a raw source node address, the destination node 

packet data buffer in accordance with the present invention. ^; "^^f^f ^ ^V^^ ^ ^ 
. . 1 J. Ml ^ *^ 1 dn coupled to a transmission medium 205 for a network seg- 

FIG. 5c IS a block diagram tU«»tratmg aD «en;ptary « ^ent through the network interface 240. TTie network into- 

erabodiment of a daU table denvcd from daU u, a decoded J ^ ^ ^ ^^^^ 

packet data buCte in accordance with the present invenUon ^^^^^ nkwork media to which it is attached. The 

FIG. 6a and 66 are lock diagrams Ulustoting a prefeired ^^^^^ .^^^^^^ ^ ^ daU to the 

embodimemofadautableforstormgpactetinfoniiationto packet capture section 230. lUe packet capture section 230 

recompile packet data m accordance with the present inven- • ^ ^^^^^ to filter out any undesired raw packets and to 

hold the desired packet data in a buffer for analysis. 

FIG. 6c is a block diagram iUuswting a preferred cirfxxli- ^ ^ ^ ^.^ exchange between many 

ment of a data uble in a raw transaction data buffer m ^^^^ ^ ^ ^^^^^ ^ 

accordance with the present invention, „„„i„„:e o^a c^, *u-. 

^ ^ . . ^ ,^ . 50 analysis section 220. For example, the user may seek to 

FIG. 6d is a block diagram further illustrating a field m a ^^p^„^^ p^^j^^^ ^ ^^^^ jhc cxdiange of 

raw transaction data buffer data table, information between five Internet sites and, therefore, all 

FIG. 7 is a block diagram illustrating a preferred embodi- jjodes which oonmiunicatc with them. Preferably, the packet 

mentof a data table in a translated transaction data buffer in capture section 230 will filter the packets traversing the 

accordance with the present invention, network medium so as to capture only packets to or from the 

FIG. 8 is a flow chart illustrating a preferred embodiment nodes of interest. The packet data is further decoded and 

of a method for analyzing internet activity in accordance recompiled by the packet analysis section 220 to provide 

with the present invention. con^jrehensible data and to allow its review and manipula- 

FIG. 9 is a flow chart illustrating a preferred embodiment tion. For exaiiq)le, the packet analysis section 220 preferably 

of a method for capturing raw pack data in accordance with sorts the packets by nodes in a transaction and uses infor- 

the present invention. mation provided in the packet to decipher a proper packet 

FIG. 10 is a flow chart illustrating a preferred embodiment exchange sequence, 

of a method for recompiling raw data packets in accordance The packet analysis section 220 receives the buffered 

with the present invention. packet data and decodes certain information in the packets 

FIG. 11 is a flow chart illustrating a preferred embodiment 65 to provide information sudi as the sources and destinations 

of a method for translating transactional data in accordance of the packets. Many packets are typically exchanged 

with the present invention. between two nodes to complete a task or transaction. 
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The packet analysis section 220 also translates the packet 
data according to applications/presentation layer protocols 
such as FTP, HTTP, SMTP. TELNET, and others (hereinafter 
application protocols). This allows the packet analysis sec- 
tion 220 to identify the pieces of protocol inf oimation which 5 
are used by the nodes In the transaaion to instruct them what 
to do. These actions do not need to be fully simulated. 
Rather, merely enough information to enable the extraction 
of pertinent data regarding the instructions and to collect 
statistical information about them needs to be gadiered. lo 

The data management and storage section 210 is coupled 
to the packet analysis sectioo 220 to access the data com- 
piled and stored therein, to obtain information from the data, 
and to manage its storage. 

A variety of information may be obtained from the raw 
data, the decoded data, the recompiled data and the trans- 
lated data. For exan^ie, the number . type, sequence and 
duration of user visits to various locations may be provided. 
Additionally, inferred information regarding the way that th 
e data is exchanged may be obtained. Preferably, the data 
management and storage section 210 obtains timing and 
sequencing information regarding the exchange of data 
between two nodes for analysis. Additionally, the general 
level of network utilization required for a transaction is 
preferably calculated. Detailed information concerning 
transactions between selected nodes may be obtained by 
analysis of the decoded data, such as the type of transaction, 
the type of browser used and the efficiency of the transac- 
tion. 

30 

The data management and storage section 210 also con- 
serves data storage space by filtering the decoded data to 
linut it to a desired set and indexing the data to avoid 
redundant storage of the same data. The data management 
and storage section 210 also manages the long term storage 
of the data. 

The report generation section 250 provides utilities for 
configuring the data provided by the data management and 
storage section 210. Specifically, reports in the forms of 
gr^hs. charts, diagrams, or plain text may be provided. This ^ 
data is provided to a report and statistics retrieval section 
260 which allows local access, or remote access through the 
security interface 280, to the report data. 

The increasing use and reliance upon the Internet has also 
increased the need for reliability. Therefore, it has become 43 
important for managers of Internet sites and Internet Service 
Providers to know immediately of problems or crashes on a 
network or servers on the network. The alarm generation 
section 270 accesses data in Che packet analysis section 220 
provides a rule based system for. inter alia, notification to ^ 
selected individuals where traffic levels for the entire net- 
work media exceeds predetermined thresholds or is below 
other thresholds. Additionally, the alarm generation section 
270 may be configured to monitor trafBc over preselected 
nodes of interest The alarm generation section provides the 
data to the selected individuals through the security interface 
280, whidi may be configured accordingly. 

Finally, access lo the internet activity analyzer 200 may be 
provided through a security interface 280 to maintain, where 
necessary, confidentiality. Conventional login procedures 60 
may be provided to restrict access and tampering. The 
collected data may be encrypted as a further barrier to 
access. 

Referring now to FIG. 3, a block diagram of a preferred 
embodiment of the internet activity analyzer 300 in accor- 65 
dance with the present invention is shown to comprise a 
display device 304, a central processing unit (CPU) 306, a 
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memory 308, a remote access interface 310. an input device 
312, a data storage device such as a hard disk 314 and a 
network interface 316. The CPU 306 is connected by a bus 
302 to the display device 304, memory 308, remote access 
interface 310, input device 312. data storage device 314 and 
network interface 316 in a von Neumann architecture. The 
CPU 306 is preferably a microprocessor such as a Motorola 
Power PC series type processor, the display device 304 is 
preferably a video monitor, the data storage device 314 is 
preferably a hard disk, and the input device 310 preferably 
includes a keyboard and mouse. The memory 308 is pref- 
erably a random access memory (RAM), but may also be a 
read only memory (ROM) or a combination of both RAM 
and ROM. The CPU 306, input device 312. display device 
304. data storage device 314 and memory 308 may be 
arranged to communicate in a conventional manner such as 
in a personal computer but it is understood that the internet 
activity analyzer 300 may be, for example, a mainframe 
computer. It is also understood that the CPU may be other 
processor types such as a Pentium processor. Alternatively, 
the local keyboard and mouse for the input device 312 and 
local display device 304 may be omitted to provide a "black 
box*' type of activity analyzer which may be accessed 
remotely by those parties with the appropriate access infor- 
mation. 

The network interface 316 is coupled to the bus 302. and, 
via a conventional cable or line 320, to a network transmis- 
sion medium to connect the internet activity analyzer 300 
into a selected network segment. The internet activity ana- 
lyzer 300 is preferably connected into a segment which 
includes one or more targeted nodes or services con^sing 
a plurality of nodes. For example, the internet activity 
analyzer 300 may be connected into the segment in which 
the world wide web server for netscape com resides in order 
to record and report usage statistics regarding the server 
generally and. for example, for eadi page on the server. 
AltematiYely. the internet activity analyzer 300 may be 
connected to a segment that selected users are on in order to 
monitcH" their usage of the Internet or to a segment repre- 
senting part of the Intern^ backbone in order to monitor its 
performance and usage. Additionally, a plurality of internet 
activity analyzers 300 may be connected to network seg- 
ments to access more comprehensive information. Although 
the internet activity analyzer 300 is primarily intended for 
analyzing internet activity, including Aat on the Internet, is 
understood that the analyzer 300 may be used to monitor any 
network, including a local area network (LAN) if desired. 

The network interface 316 is preferably a network inter- 
face controller (NIC) as manufactured, for example, by 
National Semiconductor for interface to network media. The 
receiver portion of the network interface 316 is preferably 
configured to receive all packets traversing the n^ork 
segment by using the NIC promiscuous [^ysical capability. 
In the promiscuous physical mode, the receipt of packets by 
the network interface 316 is transparent, and, thus, allows 
packet capture without disturbing the exchange between the 
source and destination nodes. The packet data captured by 
the network interface 316 is output to the system bus 302 for 
processing by the analyzer 300. 

The remote access interface 310 is coupled to the bus 302 
and to an outside line 318 such as a telephone line to provide 
alternative access to the analyzer 300. The remote access 
interface 310 is preferably a modem but may be any means 
for communicating the data produced by the analyzer 300. 
The remojte access interface 310 may be configured to notify 
selected parties where predetermined conditions are 
detcaed by the analyzer 300. Such notification may be. for 
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example, provided via pager alarms or telephone calls using 
computer generated voices, Alteraatively. the remote access 
interface 310 may be a network interface connected to a 
network segment for Internet access to the data provided by 
the analyzer 300. All data accesses may be secured, 
encrypted or otherwise limited to selected parties. 

The CPU 306. under the instructions received from the 
memory 308 as configured by the user through the input 
device 312, provides signals for processing the packets 
which traverse the network medium 320. For example, die 
CPU 306, as instructed by the memory 308, may initiate the 
capture of packets by the network interface 316, store and 
process the packet data in the memory 308, display selected 
data locally via the display device 304, store the data for 
future use in the data storage device 314, or communicate 
with remote entities through the remote access interface 310. 

Referring now to FIG. 4a, a preferred embodiment of the 
internet activity analyzer 300 memory 308 is shown in more 
detail. The memory 308 preferably coix^rises a packet 
capturing module 322, a packet analyzer 324. a data man- 
agement module, a ou^ut configuration module 328 and an 
alarm generator 330. TTie internet activity analyzer 300 
preferably uses a conventional operating system such as 
Mac OS but the artisan will recognize that a variety of 
alternative operating systems may be implemented such as 
Wndows or UNIX. 

The packet capturing module 322 communicates with the 
network interface 316 through die system bus 302 and 
includes routines which capture and hold raw packets of data 
for processing by the analyzer 300. The packet capturing 
module 322 is shown and described in more detail with 
reference to FIG- 4i?. 

The packet analyzer 324 receives the raw packet data 
through the system bus 302 and includes routines which 
decode the raw packet data into coherent infonnation such 
as that regarding its source and destination, recompiles data 
from a plurality of packets from exchanges between nodes 
into raw transaction data and translates the data in the 
packets at the applications level to identify instructions b 
the transactions between the nodes. A single exdiange 
between first and second nodes may be c^tured and ana- 
lyzed by the packet analyzer 324. However, the packet 
analyzer 324 is arranged to process a plurality of exchanges 
between a plurality of nodes. For example, a first transaction 
including a plurality of exchanged packets between first and 
second nodes and a second transaction including a plurality 
of exchanged packets between third and fourth nodes may 
be simultaneously processed. The packet analyzer 324 is 
shown and described in more detail with reference to FIG. 
4c. 

The data management module 326 accesses the informa- 
tiOD in the raw packet decoded packet, raw transaction and 
translated transaction data buffers through the system bus 
302 and includes routines for inferring network infonnation 
about the data such as that related to timing and sequencing, 
generating information about the data, indexing the 
information, and managing data storage. The data manage- 
ment module 326 is shown and described in more detail with 
reference to FIG. Ad. 

The ouQ)ut configuration module 328 communicates with 
the data management module 326. input device 312 and 
remote access interface 310 and includes routines for con- 
figuring data reports from the data management module 326. 
The data report routines may be conventional programs for 
the manipulation of data into display formats such as gr^s, 
charts or diagrams and may be customized according to each 
particular application. 
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The alarm generator 330 communicates with the data 
management module 326 and is configured to issue alarms 
based upon the infonnation contained therein. Preferably, 
the alarm generator 330 includes threshold information, data 

5 defining the scope of interrogation, and data identifying 
parties to notify in the event of alarm conditions. These 
values allow the alarm generator 330 to monitor the data in 
the data management module 326 and to notify identified 
parties through the remote access interface 310. 

10 Referring now to FIG. 4fr, a block diagram of a preferred 
embodiment of the packet capturing module 322 is shown in 
more detail to include a raw packet filter 332 and a raw 
packet data buffer 334. The packet capturing module 322 
communicates with the system bus 302 to receive the raw 

15 packet data which is output by the network interface 316. 
Alternatively, raw packet data may be provided to the 
analyzer 300 by other sources through the remote interface 
310. The raw packet data includes all of the packets tra- 
versing the network media to which the analyzer 300 is 

^ attached. This data can be voluminous, and, where 
necessary, the raw packet filter 332 is provided to filter the 
data into a manageable set. 

Referring now to FIG. 5a, a block diagram of a simplified 
conventional raw packet data profile 500 is shown to com- 
prise a local network header 505, an internet protocol (IP) 
header 510, a transmission control protocol (TCP) header 
515, application data 520 and a local network trailer 525. 
The raw packet filter 332 is in communication with other 
modules such as the IP packet decoder 336 to facilitate the 
filtering of packets. The packets may be filtered by a variety 
of data provided in the packet data profile 500. but. 
preferably, the raw packet filter 332 is set to output data 
packets for which selected nodes are the source or destina- 
tion. This facilitates examination of the exchange of dau 
between the selected nodes and any other nodes, whether the 
other nodes are selected or not For example two exchanges 
between first through fourth nodes may be captured. To 
facilitate such filtering, the raw packet filter 332, in con- 
junction with, for example, the IP packet decoder 336, 

^ decodes a portion of eadi packet traversing the network 
medium to which the analyzer 300 is attached. For example 
the IP source and destination address poitions of the data 
profile 500 may be determined by the raw packet filter 332 
to ascertain the raw packet source and destination addresses. 
This data may then be coii:q>ared to predetermined address 
data stored in memory 308. If the predetermined address 
data matches, fc^' exan^le, the source or destination address 
data in the packet the filter 332 stores the packet in the raw 
packet data buffer 334. 

Preferably, the predetermined data is stored in memory 
308 before q>eration, but comparison data may be provided 
by the user through the input device 312 <x via the remote 
access interface 310 for real time analysis. 

55 Although filtering raw packets by source or destination 
address has been described, it is understood that the packet 
capturing module 322 may be arranged to filter by other 
criteria such as by the protocol which describes a service 
such as FTP or HTTP and that the criteria for filtering may 

50 be found in the any available field in the raw packet data 
profile 500. Additionally, the artisan will recognize that the 
internet activity analyzer 300 may be adapted to treat data 
transmitted according to protocols other than that shown in 
FIG. Sa sudj as the user datagram protocol (UDP) in lieu of 

65 TCP 

Referring again to FIG. 4b, the retained raw packets are 
provided to the raw packet data buffer 334 for access to and 
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further processing by the analyzer 300. The raw packet data 
bufifer 334 is sized according to the level of traffic on the 
network segment and the expected level of filtering. 
Preferably, the raw packet data buflFer 334 is a data FIFO 
with sufficient capacity for the expected quantity of data. 5 

Referring now to FIG. Sb, a sample data table 550 derived 
from data provided in the raw packet data buffer 334 is 
shown to include packet identity 555, source physical 557, 
destination physical 559, size 559 and time stamp 561 fields 
for a plurality of sample entries. Specifically, the table in 10 
FIG. 5b shows a list of packets received promiscuously 
between two locations. The physical source 557 and desti- 
nation 559 addresses are extracted from the local network 
header 505 in each packet as is inferred data about the size 
and time of arrival of the packet. The inferred data may be i^ 
provided, for example, as described in connection with the 
data management module 326 below. Such data provides 
usage and performance information which may be jffovided 
to the alarm generator 330. 

Refeiring now to FIG. 4c, a block diagram of a preferred 
embodiment of the packet analyzer 324 is shown in more 
detail to include a transmission control protocol internet 
protocol (IP) packet decoder 336, a decoded packet data 
buffer 338. a data sorter 340. a decoded packet reconopiler 
341, a raw transaction data buffer 342, a set of filters 344, an 
application protocol translator 346 and a translated transac- 
tion data buffer 348. 

Preferably, the packet analyzer 324 communicates with 
the raw packet data buffer 334 of the packet capturing ^ 
module 322 to receive raw packets. For example, raw 
packets may be provided to the IP packet decoder 336 which 
uses conventional IP (internet protocol) decoding techniques 
to convert the information in each packet to decoded data 
sudi as that provided, for example, in the local network 
header 505 the IP header 510 and the TCP header 515. 

This inf(Hination includes administrative data for identi- 
fying and organizing packet exchanges as well as informa- 
tion identifying the source and destination at various layers. 
For example, the local network header 505 includes local ^ 
network source and destination addresses, the IP header 510 
includes source and destination internet protocol addresses, 
which identify the actual nodes conmiunicating with each 
other, and the TCP header 515 includes source and destina- 
tion port addresses, sequencing and acknowledgment infor- 
mation and raw (untranslated) application data. The IP 
packet decoder 336 provides the decoded packets to the 
decoded packet data buffer 338 for access to and further 
processing by the analyzer 300. 

Refeiring now to FIG. 5c, an entry 570 for a sair^le data 50 
table 569 derived from data provided in the decoded packet 
data buffer 338 is shown to include a packet identifying field 
571 as well as plural fields for the packet date and time 
stamp and other information 572, the local network (such as 
ethemet) header 574, the IP header 576 and the TCP segment 55 
578 which includes a header and raw application data. 

The packet analyzer 324 data sorter 340 and filters 344 
may be used to further reduce the packet data set according 
to the fields provided in the decoded packet data buffer 338. 
For example, if an undeslred IP port repeatedly appears in 60 
the decoded packet data, the user may. through the input 
device 312 dkectly or via the remote access interface 310, 
remove packets from the decoded packet data buffer 338 
which include the IP port as a TCP source or destination. 
Alternatively, data for seating, comparison and filtering may 6S 
be provided beforehand such as in the noted *l)lack box** 
configuration. 
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The decoded packet recompiler 341 accesses the decoded 
packet data buffer 338 to recompile the packets into coherent 
transaction data. As indicated, a transaction between two 
nodes typically comprises a plurality of packet exchanges. 
Since most packets must be acknowledged to ensure trans- 
actional integrity, packets are often retransmitted where the 
lag between the transmission of the packet and its acknowl- 
edgment exceeds a predetermined value. Additionally, 
where packets are iir^roperly received oc lost, they may be 
retransmitted out of sequence with other packets in the 
transaction. Thus, the set of captured packets for a transac- 
tion will often be out of sequence and include duplication. 
The plural packets are properly sequenced via an analysis of 
the administrative data included in the packet headers such 
as the sequencing and acknowledgment infcaination pro- 
vided in the TCP segment 515, 578. Duplicate packets are 
discarded after sequencing. Most of the administrative data 
may then be discarded and the raw application data from the 
sequential packets may be coherently concatenated. 

The decoded packet reconqjiler 341, in conjunction with 
the data sorter 340. the filters 344, and data in the decoded 
packet data buffer 338, is arranged to recompile transactions 
between plural nodes. Referring now to FIG. 6a. a block 
diagram of a data table 600 is shown to comprise packet 
identifier 605, source port identity 610. destination port 
identity 615, source IP address 612 and destination IP 614 
address fields. This data may be accessed from the decoded 
padcet data buffer 338 from fields residing in the IP header 
510. 576 and the TCP segment 515, 578. In Ae example 
shown in FIG. 6a, exchanges between 8 nodes is shown. 
Eleven entries far the packet exchanges are shown, but it is 
understood that many more packet entries may be provided. 
To reoon:^5ile the various packets into coherent raw trans- 
action data, the decoded packet recompiler 341 must first 
separate the packets into conversations between nodes. In 
this instance, suppose the user wants to analyze data regard- 
ing any exdianges in whidi source port 80 was a participant 
The filters 344 may be applied to the data to remove packets 
which do not include port 80 as either a source destina- 
tion. Alternatively, such filtering may be undertaken by the 
packet capturing module 322 using predetermined data. 
Thus, in the shown example, the rows corresponding to 
packet numbers 4 and 8 may be removed from the data table 
600. The data in the table 600 may then be sorted to 
segregate the data into exchanges between selected nodes. 
First, the data may be sorted by nodes in a transaction and 
then each transaction may be recompiled into the proper 
sequence. Preferably, hashing techniques may be used to 
group the data, but the artisan will recognize alternative 
techniques. 

Referring now to FIG. 66, the data table 600 is shown to 
exclude packets from exchanges which do not include the 
selected ports. Additionally, the entries arc segregated 
according to the packets exchanged between the selected 
ports. For example, packet entries 1, 11, 3, 5, and 7 con:q3rise 
the exchange between ports 80 and 1842 (nodes 1 and 2) and 
packet entries 2, 10 and 6 comprise the exchange between 
ports 80 and 1827 (nodes 3 and 4). The information in the 
packet identity fields 605 preferably corresponds to a packet 
identity field in the decoded packet data buffer 338 so that, 
after properly segregating the packet exdianges using only 
the necessary data, the application data fields 520 for the 
packets may be recompiled in sequence to provide raw 
transaction data. Additionally, by using corresponding 
packet identities, other data in the decoded packet data 
buffer 338 such as the IP source and destination addresses in 
the IP header 510» 576 may be accessed by the decoded 
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packet recompiler 341 to provide information to the raw 
transaction data buffer 342. 

Referring now to FIG. 6c, a sample data table 650 in the 
raw transaction data buffer 342 is shown to include entry 
identification 655. first and second node identification 660* 
665 and data 670 fields. The data field 670 includes coher- 
ently concatenated data; however, the data remains untrans- 
lated. Again, the information in the raw transaction data 
buflfer 342 may be accessed by the packet analyzer 324 data 
sorter 340 and filters 344 to manipulate the data. 

Referring now to FIG. 6d, for illustrative purposes, some 
entries 685, 690 in a data field 670 in a raw traosaction data 
buflfer 342 data table 650 are shown to include a portion of 
concatenated raw transactional data in text 675 and hex 680. 

Referring again to FIG. 4c, the application protocol 
translator 346 preferably communicates with the raw trans- 
action data buffer 342 to access the reconciled raw trans- 
action data 670. The application protocol translator 346 
includes routines for decoding International Organization 
for Standardization (ISO) application/presentation layer 
transfers such as electronic mail and file transfers. For 
example, the application protocol translator 346 may include 
file transfer protocol (FTP) routines to decode infcHination 
regarding file transfers between nodes, hypertext transfer 
prcHocol (HTTP) routines, or other well known application 
layer protocols. The instructions defined by the transaction 
data do not have to be completely simulated to provide 
relevant information regarding the transactioQ. 

I^referably, to extract pertinent data without unduly con- 
suming resources, lexical stream parsing of the transactional 
data is undertaken such diat keyword and value analysis of 
the data may be undertaken to extract die relevant data. First, 
the translator 346 determines what fields to find in the 
transactional data and, siraQarly, determines the appropriate 
keyword for scanning for the field. The selected fields could 
be provided in memory 308 or may be provided through the 
input device 312 by the user. Then, the transactional data is 
scanned by the translator 346 to find the keywOTd. access the 
value associated with it, and store that value in the appro- 
priate field of a data table 700 in the translated transaction 
data buffer 348. For exan^>le, suppose the user wanted to 
scan for the browser used. The HTTP protocol routines, as 
provided in the aj^Ucation protocol translator 346, recog- 
nize that the value following the "User-Agent** keyword 
includes the appropriate value. Referring to the first entry 
685 in the data field 670 of FIG. the translator would 
scan through the data, recognize the "User- Agent" keyword, 
and access the value *TCP InterconnectlT as the value 
associated with it. The application protocol translator 346 
can then provide this information for storage in the trans- 
lated transaction data buffer 348. 

Referring now to FIG. 7, a sample data table 700 in the 
translated transaction data buffer 348 is shown to include 
entry identification 705, first and second identification 710. 
715, bytes transfeircd 720, transaction time 725 and browser 
used 730 fields. Of course, a great number of entries may be 
provided in the data table to provide additional information. 

Referring now to FIG. 4J, a block diagram of a preferred 
embodiment of the data management module 326 is shown 
in more detail to include a logical unit 350, a data indexing 
and stripping module 352, an indexed data buffer 356, an 
inference analyzer 358 a data buffer 360 and a sorting and 
filtering module 362. The data management module 326 
accesses data in the raw packet 334, decoded packet 338, 
raw transaction 342 and translated transaction 348 data 
buffers to provide real time, inferred and customized data to 
the user. 
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Real time data can be analyzed by accessing the various 
data buffers 334, 338, 342, 348 directly through the input 
device 312 or via the remote access interface 310. The 
observed data may be customized by the data management 

5 module 326 and the output configuration module 328. 

The logical unit 350 can access timing data in any of the 
data buffers 334, 338, 342, 348, store it in the data buffer 
360, and manipulate it to acquire information about the 
timing and sequence and related performance of the 
network, die nodes, or other entities. For example, a bar 
graph indicating usage of the network media relative to 
100% capacity may be observed. To provide such data, data 
representing the maximum bandwidth capabilities of die 
network may be stwed in memory 308. Typically this data 

^5 is in the form of a number of bits per second. The number 
of bits traversing the network medium may be calculated 
from data provided through the network interface 316 and 
this number is coimted over a given time period. The actual 
rate is compared to the bandwidth capability data to provide 

2Q the relative use data. A separate clocldng device (not shown) 
may be provided in the internet activity analyzer 300 where 
very high accuracy is desired. 

Additionally, die data management module 326 may be 
airanged to rate Internet usage for one or more selected sites. 

25 Preferably, to rate such usage, the raw packet filter 332 is 
arranged to retain only those packets which include the site 
node address as a source or destination. Alternatively, raw 
packets could be c^tured for additional sites and the data 
could be filtered, after decoding, recompilation. or transla- 

30 tion by the packet analyzer filters 344. Thus, the information 
in the translated transaction data buffer 348 could be limited 
to entries for exchanges in which the site was a participant 
The logical unit 350 can provide a raw count of such entries 
to rate overall usage or provide more detailed Infcrmation 

35 such as the browsers used to access the site, the average time 
spent, the average time used for selected transactions, a time 
to byte transferred ratio. Additionally, information regarding 
hits and data on individual pages may be provided. 
The inference analyzer 3^ may also extract inferred data. 

40 preferably from the information in the translated transaction 
data buffer 348. For example, a "GET" command is a 
conmiand in the HTTP protocol fi-om a browser to a server, 
so the nodes in a transaction may be identified as such. This 
allows the data to be segregated accordingly and, likewise. 

43 allows the analyzer 300 to focus on browser or server 
statistics where desired. Additionally, by communicating 
widi the Internet through, for example, the remote access 
interface 310, information about nodes can be determined. 
Specifically, suppose a node is generating activity fcF which 

so inquiry is desired. By transmitting a reverse DNS (Oomain 
Name Service) Query, the IP address field (typically a string 
of numbers) can be used to provide its Internet address. 
Thus, an e-mail or other warning could be sent to that user 
Additionally, by accessing the IP addresses of the users that 

55 visit a site and using the reverse DNS lookup, visitors can be 
classified in terms of country, conmiercial, educational, or 
government Moreover, using readily available Internet 
resources, a variety of information beyond the domain name 
may be ascertained, 

60 Adxlitionally, the data sets may be manipulated for long 
term st(»rage by the data indexing and stripping module 352 
and the indexed data buffer 356. The indexing and stripping 
module uses indexing algorithms to recognize previously 
handled quantums of data and instead of repeating storage in 

65 memory or the data storage device, the data may be indexed 
to a common storage area using, for example, a shorthand 
number which consumes a minimal number of bytes used in 
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place of a longer, frequently replicated piece of data. activity analyzer 2(K). ^00 and that the data set, in partial or 

Additionally, js-ior to the creation of die index by the module in whole, may be accessed, manipulated, used to generate 

352, the data may be stripped so that detailed information information or otherwise processed at such stages, 

may be stored, for example, in the form of a shorthand Referring now to FIG. 9. a flow chart illustrating a 
number. The information may be kept in the data manage- 3 preferred method of capturing 800 raw packets is shown, 

ment data buffer or provided to the data storage device for The next packet passing through the network transmission 

long terra storage so that Internet activity may be subse- medium 320 is received 900, foe example, by the netwa-k 

quently analyzed, interface 316 of the internet activity analyzer 300. Initially. 

WhUe the flow of data has been described scquentiaUy in " 'I determined 910 whether any raw packet filters are 
order to clearly indicate how each process is undertaken, it w enabled and, if none are the raw packet is stored 945 m the 

is understood that the processes described regarding the l.'L!' packet data buffer 334 of the packet captunng module 

analyzer 300 modules 322. 324. 326. 328. 330 may occur in ^22. If any fUters are enabled, the type of filtermg desired .s 

various sequences on selected data and that the modules 322. ««rta."f 1- ^ " » detanuned 915 that transacuonal node 

324. 326. 328 and 330 may interact during packet process- «««nng is enabled flien the identity ofthe node or nodes of 

ing. For example, the IP packet decoder 336 may partially " ""^"l'^^, fof «araple. from memory 308 for 

decode packets to provide enough information to the packet co-npanson to the date m the packet to determine whether to 

recompiler 341 to construct concatenated raw transaction " buffer First the node identity data is compared 

to the source node data in the packet, and, upon a positive 

. „^ ^ , comparison, the packet is stored 905 in the raw packet data 

Referrmg now to FIG. 8, a flow chart illustmting a ^^^^ 334 ^^^^ j,^^ ^^^^ transmission 

preferred embodiment of a method for analyzing internet ^^^^ ^^^^.^^^ 5^ 

activity IB aca>rdance with the present mvcntion is shown. ^ destination data is 

Raw packet data may be captured 800 b>^ for example, compared 930 to die node identity data. SimUarly, where a 

oonnectmg the internet acUvi^ analyzer 300 to a network ^ ^^^^ j^^^ ^^^^^^ 905 in the packet data 

transmission medium 205. 320 for a selected net^^^^ ^^^^^ ^^^^ ^^^^ traversing the network 

ment. THe raw packet data may be limited to a selected data t^^^sniission medium is sought 900, If a match is not found 

set as determmed, for examp e by the raw packet filter 332 ^ comparison 930. the packet is discarded since neither 

m thepacketcapturmgmodule3^. Amore ^^^^ destination equals the node identity data. If 

of^cketcaptuieSOOisshownanddescnbedregardrngHG. transactional node filtering is not sought, then alternative 

9 below. filtering modes may be in^jlemented 935. For example, the 

The captured raw packets are then decoded 805 to provide packets may be filtered by protocol such as HTTP, FTP, 

information regarding the exchange of packets between sMTp if packets concerning sudi exdianges are sought 

nodes through the transmission medium 205, 320. The Similarly to transactional node filtering, the filtering mode is 

captured raw packets may be decoded, for example, in the IP determined, the comparison data is accessed and stored in 

packet decoder 336 of the packet analyzer 324. The raw memory 308 and the information in the packet is conq)arcd 

packet data input to the IP packet decoder 336 is decoded juch data to determine whether to store 905 the packet in 

805 to provide information such as administrative data the raw packet data buffer 334 or proceed to receive 900 the 

regarding the timing and sequence of the packets exchanged jje^t packet. It is understood that filtering according to 

between nodes. The decoded data also includes infwmation multiple fields may also be undertaken. For example, all of 
such as die IP source and destination addresses of the nodes ^ the HTTP jx-otocol exchanges between IP addresses 1. 2, 3, 

in the exchange as well as the TCP port addresses involved 4 ^nd 5 may be sought. 

in the packet exchange. Referring now to FIG, 10, a now chart illustrating a 
After decoding 805. the data may be recompiled 810 into preferred method of recompiling 810 raw packets is shown, 
raw transactional data. The packets may be recompiled, for The decoded packet data is accessed, preferably in the 
example, in the decoded packet rccompiler 341 ofthe packet 45 decoded packet data buffer 338. and, after optionally sorting 
analyzer 324. A transaction between two nodes typically 340 and filtering 344 the data, a set of data packets are 
involves a plurality of packet exchanges between those selected 1005 for recompilation. Several transactions from 
nodes which may be presented out of sequence or in many nodes may be presented in the set of packets. For 
duplicate. By accessing the administrative timing and example, a first transaction between first and second nodes 
sequencing data in the decoded packets, the packets are and a second transaction between third and fourth nodes 
reassembled into a proper sequence and the duplicates arc may be presented. The set of packet data is segregated 1010 
stripped from the data set Recon^iilation 810 produces a separate transactions by the decoded packet recompiler 
concatenated stream of raw application data which may be 341 by sorting the data, using the data sorter 340, by nodes 
stOTcd for further processing. A more detailed mediod of in a transaction. Thereafter, the packets from each transac- 
packet recompiladon 810 where plural transactions are 35 tion are arranged in a coherent sequence 1015 by the 
analyzed is shown and described regarding FIG. 10 below. recompiler 341, as indicated, through analysis of the admin- 
After recompilation 810, the data may be translated 815 istrative and other data in the decoded packets. If necessary, 
at the application level to provide information sudi as the duplicate packets may be deleted from the sequence. The 
number of bytes transferred, the type of CPU and browser raw transactional data from the data packets is therefore 
used, and other information regarding the transaction. The ^ properly arranged so that the raw data may be concatenated 
raw transactional data may be translated 815, for example, 1020 into a coherent stream for storage 1025. foe example, 
in the application protocol translator 346 in the packet in the raw transaction data buffer 342. 
analyzer 324. A more detailed method of transactional data Referring now to FIG. 11. the information from the raw 
translation 815 is shown and described regarding FIG. 11 transaction data buffer 342 is then translated 815 into more 
^iovf, 55 meaningful data. First the raw transactional data is accessed 
It is understood that FIG. 8 discloses the flow and status 1105 by the application protocol translator 346 for process- 
of data at various stages in its processing by the internet ing. The application protocol translator 346 includes appli- 
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cation level protocols such as FTP. HTTP and SMTP so that 
the concatenated data may be translated. The transaction 
between the two nodes does not need to be fully simulated. 
Pertinent data may effectively be extracted using keyword 
and value searching as described regarding the operation of 
the application protocol translator 346. First the fields to be 
scanned are identified 1110. The protocol translator 346 
converts these fields to keyword data particular to each 
application layer protocol. Then, the raw transactional data 
is scanned 1115 by keyword to locale the fields. The values 
associated with those fields are determined 1120. accessed 
and stored 1125 for exan^le, in a data table 700 in the 
translated transaction data buffer 348 for further processing. 

While the present invention has been described with 
reference to certain preferred embodiments, those skilled in 
the art will recognize that various modification may be 
provided. For example, the raw packet 334, decoded packet 
338. raw transaction 342. translated transaction 348 and 
other 360 data buffers may be provided in a centralized 
database. These and other variations upon and modifications 
to the preferred embodiment are provided for by the present 
invention which is limited only by the following claims. 

We claim: 

1. An apparatus for analyzing internet activity, the sppsi- 
ratus comprising: 

a packet capturing module, for accessing the packets 
traversing a network, the packets having source and 
destination addresses other than an address correspond- 
ing to the apparatus, and for filtering the packets to 
produce raw packet data, wherein the packet capturing 
module produces the raw packet data by retrieving a 
predetermined address, comparing the predetermined 
address to the internet protocol source address for a 
current packet, comparing the predetermined address to 
the internet protocol destination address for the current 
packet, and retaining the current packet where one of 
the internet protocol source and destination addresses 
for the current packet matches the predetermined 
address; 

a packet analyzing module, in communication with the 
packet capturing module, for producing decoded pack^ 
data and for producing transaction data from the 
decoded packet data; and 

a data management module, in communication with the 
packet capturing module and the packet analyzing 
module, for analyzing at least one of the raw packet 
data, the decoded packet data and the transaction data 
to provide an indication of internet usage. 

2. An apparatus for analyzing internet activity, the appa- 
ratus comprising: 

a packet capturing module, for accessing the packets 
traversing a n^ork. the packets having source and 
destination addresses other than an address correspond- 
ing to the apparatus, and for filtering the packets to 
produce raw packet data, wherein the packet captining 
module produces the raw packet data by retrieving a 
predetermined port^ address, comparing the predeter- 
mined port address to the transmission control protocol 
source port address for a current packet, comparing the 
predetermined port address to the transmission control 
protocol destination port address for the current packeu 
and retaining the current packet where one of the 
transmission control protocol source and destination 
port addresses for the current packet matdies the pre- 
determined port address; 

a packet analyzing module, in communication with the 
packet capturing module, for producing decoded packet 
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data and for producing transaction data from the 
decoded packet data; and 
a data management module, in communication with the 
packet capturing module and the packet analyzing 
5 module, for analyzing at least one of the raw packet 
data, the decoded packet data and the transaction data 
to provide an indication of internet usage. 

3. An apparatus for analyzing internet activity, the appa- 
ratus comprising: 

a packet capturing module, fw accessing the packets 
traversing a network, the packets having source and 
destination addresses other than an address correspond- 
ing to the apparatus, and for filtering the packets to 
produce raw packet data; 

a packet analyzing module, in communication with the 
packet capturing module, for producing decoded packet 
data and for producing transaction data from the 
decoded packet data, the packet analyzing module 
con^sing: 

a packet decoder, for accessing the raw packet data and 
producing the decoded packet data; and 

a decoded packet reconciler, in conmiunication with 
the packet decoder, for accessing the decoded packet 
2^ data, segregating the packets from the decoded 

packet data into separate transactions between nodes, 
sequencing the packets corresponding to each sepa- 
rate transaction, and concatenating the data in each 
separate transaction to produce the transaction data; 

30 

a data management module, in communication with the 
packet capturing module and the packet analyzing 
module, for analyzing at least one of the raw packet 
data, the decoded packet data and the transaction data 
35 to provide an indication of internet usage. 

4. The apparatus of claim 3. wherein the transaction data 
includes a first transaction and a second transaction, the first 
transaction comprising packets exchanged between a first 
node and a second node and the second transaction com- 

^ prising packets exdianged between a third node and a fourth 
node. 

5. An apparatus for analyzing internet activity, the ^pa- 
ratus comprising: 

a packet capturing module, for accessing the packets 
43 traversing a netwo-k, the packets having source and 
destination addresses other than an address correspond- 
ing to the apparatus, and for filtering the packets to 
produce raw packet data; 
a packet analyzing module, in communication with the 
30 packet capturing module, for producing decoded packet 
data, for producing transaction data from the decoded 
padcet data, and fcr {x-odudng translated transaction 
data from the transaction data, the packet analyzing 
module comprising: 
55 a packet decoder, for accessing the raw packet data and 
producing the decoded packet data; 
a decoded packet recompiler, in conununication with 
the packet decoder, for accessing the decoded packet 
data, segregating the packets from the decoded 
60 packet data into separate transactions between nodes, 

sequencing the packets corresponding to each sepa- 
rate transaction, and concatenating the data in each 
separate transaction to produce the transaction data; 
and 

65 an ^plication protocol translator, in communication 
with the decoded packet recompiler. for producing 
the translated transaction data by accessing the trans- 
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action data, scanning the transaction data for a field 
corresponding to a selected application jx-otocol. 
determining a value associated with the Held and 
storing the field and the associated value; and 
a data management module, in communication with the 5 
packet capturing module and the packet analyzing 
module, for analyzing at least one of the raw packet 
data, the decoded packet data, the transaction data, and 
the translated transaction data to provide an indication 
of internet usage. 

6. The apparatus of claim 5, wherein the transaction data 
includes a first transaction and a second transaction, the first 
transaction comprising packets exchanged between a first 
node and a second node and the second transaction com- 
prising packets exchanged between a third node and a fourth 15 
node. 

7. The apparatus of claim 5, wherein the data management 
module produces a profile of a selected site by determining 
the number of transactions in which the selected site par- 
ticipates, 20 

8. The apparatus of claim 7, wherein the data management 
module produces analytical data corresponding to selected 
transactions that the selected site participates in. the ana- 
lytical data for each selected transaction including at least 
one of the browser type used to access the selected site, the 25 
amount of data tran^eired in the transaction, and the trans- 
action time. 

9. The apparatus of claim 7. wherein the data management 
module accesses the addresses for nodes transacting with the 
selected site, uses a reverse domain name service lookup to ^ 
obtain information about the nodes, and uses the information 
about the nodes to produce the profile of the seleaed site. 

10. For use with an internet activity analyzer capable of 
being coupled to a network transmission medium, a method 

of analyzing internet activity, the method comprising: 35 
accessing the packets traversing the network, the packets 
having source and destination addresses other than an 
address corresponding to the internet activity analyzer; 
filtering the packets to produce raw packet data by retriev- 
ing a predetermined address; comparing the prcdeter- ^ 
mined address to the internet protocol source address 
for a cunent packet; con^jaring the predetermined 
address to the internet protocol destination address for 
the current packet; and retaining the current packet 
where one of the internet protocol source and destina- 
tion addresses for the current packet matches the pre- 
determined address; 
producing decoded packet data; 

producing transaction data firom the decoded packet data; ^ 
and 

analyzing at least one of the raw packet data« the decoded 
packet data and the transaction data to provide an 
indication of internet usage. 

11. For use with an internet activity analyzer capable of 55 
being coupled to a network transmission medium, a method 

of analyzing internet activity, the method comprising: 
accessing the packets traversing the network, the packets 
having source and destination addresses other than an 
address corresponding to the internet activity analyzer; 60 
filtering the packets to produce raw packet data by retriev- 
ing a predetermined port address; con^aring the pre- 
determined port address to the transmission control 
protocol source port address for a current packet; 
comparing the predetermined port address to the trans- 65 
mission control protocol destination port address for 
the current packet; and retaining the current packet 



where one of the transmission control protocol source 
and destination port addresses for the current packet 
matches the predetermined port address; 

producing decoded packet data; 

producing transaction data firom the decoded packet data; 
and 

analyzing at least one of the raw packet data, the decoded 
packet data and the transaction data to provide an 
indication of internet usage. 

12. For use with an internet activity analyzer capable of 
being coupled to a network transmission medium, a method 
of analyzing internet activity, the method comprising: 

accessing the packets traversing the network, the packets 
having source and destination addresses other than an 
address corresponding to the internet activity analyzer; 

filtering the packets to produce raw packet data; 

producing decoded packet data; 

producing transaction data from the decoded packet data 
by accessing the decoded packet data; segregating the 
packets from the decoded packet data into separate 
transactions between nodes; sequencing the packets 
corresponding to each separate transaction; and con- 
catenating the data in each separate transaction to 
produce the transaction data; and 

analyzing at least one of the raw packet data, the decoded 
packet data and tiie transaction data to provide an 
indication of internet usage. 

13. The method of claim 12, wherein the transaction data 
includes a first transaction and a second transaction, the first 
transaction comprising packets exchanged between a first 
node and a second node and the second transaction com- 
prising packets exchanged between a third node and a fourth 
node. 

14. For use with an internet activity analyzer capable of 
being coupled to a network transmission mediunt a method 
of analyzing internet activity, the method comprising: 

accessing the packets travening the network, the packets 
having source and destination addresses other than an 
address corresponding to the internet activity analyzer; 

filtering the packets to produce raw packet data; 

producing decoded packet data; 

producing transaction data from the decoded packet data 
by accessing the decoded packet data; segregating the 
packets from the decoded packet data into separate 
transactions between nodes; sequencing the packets 
corresponding to each separate transaction; and con- 
catenating the data in each separate transaction to 
produce the transaction data; and 

producing translated transaction data from the transaction 
data; and 

analyzing at least one of the raw packet data, the decoded 
packet data, the transaction data, and the translated 
transaction data to p-ovide an indication of internet 
usage. 

15. The method of claim 14. wherein the step of produc- 
ing translated transaction data comprises: 

accessing the transaction data; 

scanning the transaction data for a field corresponding to 

a selected application protocol; 
determioing a value associated with the field; and 
retaining an association between the field and the deter- 
mined value. 

16. The method of claim 15. wherein the transaction data 
includes a first transaction and a second transaction, the first 
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transaction comprising packets exchanged between a first 
node and a second node and the second transaction com- 
prising packets exchanged between a third node and a fourth 
node. 

17. The method of claim 16, further con^>rising: 5 
producing a profile of a selected site by determining the 

numbCT of transactions In which the selected site par- 
ticipates. 

18. The method of claim 17, wherein analytical data 
corresponding to selected transactions that the selected site 
participates in is produced, the analytical data for each 
selected transaction including at least one of: the browser 
type used to access the selected site, the amount of data 
transferred in the transaction, and the transaction time. 

19. The method of claim 17, further comprising: ^5 
accessing the addresses for nodes transacting with the 

selected site; 

using a reverse domain name service lookup to obtain 

information about the nodes; and 
using the infonnation about the nodes to produce the 

profile of the selected site. 

20. An apparatus for analyzing internet activity, the ^pa- 
ratus comprising: 

means for accessing the packets traversing the network. 25 
the packets having source and destination addresses 
other than an address corresponding to the internet 
activity analyzer; 

means for filtering the packets to produce raw packet data, 
wherein the means for filtering the packets to produce 
raw packet data includes routines for retrieving a 
predetermined address; comparing the predetermined 
address to the internet protocol source address for a 
current packet; comparing the predetermined address to 
the internet protocol destination address for the current 35 
packet; and retaining the current packet where one of 
the internet protocol source and destination addresses 
for the current packet matches the predetermined 
address ; 

means far producing decoded packet data; ^ 
means for producing transaction data from the decoded 

packet data; and 
means for analyzing at least one of the raw packet data. 

the decoded packet data and the transaction data to 45 

provide an indication of internet usage. 

21. An apparatus for analyzing internet activity, the appa- 
ratus comprising: 

means for accessing the packets traversing the network, 
the packets having source and destination addresses 



other dian an address corresponding to the internet 
activity analyzer; 

means for filtering the packets to produce raw packet data, 
wherein the means for filtering the packets to produce 
raw packet data includes routines for retrieving a 
predetermined port address; comparing the predeter- 
mined port address to the transmission control protocol 
source port address for a current packet; comparing the 
predetermined port address to the transmission control 
protocol destination port address for the current packet; 
and retaining the current packet where one of the 
transmission control protocol source and destination 
port addresses for the current packet matdies the pre- 
determined port address; 

means for producing decoded packet data; 

means for producing transaction data from the decoded 
packet data; and 

means for analyzing at least one of the raw packet data, 
the decoded packet data and the transaction data to 
provide an indication of internet usage. 

22. An apparatus for analyzing internet activity, the appa- 
ratus comprising: 

means for accessing the packets traversing the network, 
the packets having source and destination addresses 
other than an address coaresponding to the Internet 
activity analyzer; 

means for filtering the packets to (roduce raw packet data; 

means for producing decoded packet data; 

means for producing transaction data from the decoded 
packet data, wherein the means for producing transac- 
tion data includes routines for accessing the decoded 
packet data; segregating the packets from the decoded 
packet data into sq>arate transactions between nodes; 
sequencing the packets corresponding to each separate 
transaction; and concatenating the data in each separate 
transaction to i^-oduce the transaction data; and 

means for analyzing at least one of the raw packet data, 
the decoded packet data and the transaction data to 
provide an indication of internet usage. 

23. The ^aratus of claim 22. wherein the transaction 
data includes a first transaction and a second transaction, the 
first transaction comprising packets exchanged between a 
first node and a second node and the second transaction 
comprising packets exchanged between a third node and a 
fourth node. 
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